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After  40  Years  Why  Hasn’t  the  Computer  Replaced  the 

Wind  Tunnel? 

Edward  M.  Kraft,  Ph.D. 

USAF  Arnold  Engineering  Development  Center,  Arnold  Air  Force  Base,  Tennessee 

The  debate  between  wind  tunnels  and  computers  to  develop  aeronautical  systems  has  persisted 
for  over  40  years.  On  the  one  hand ,  the  majority  of  wind  tunnels  used  today  in  aeronautical 
research,  development,  test,  and  evaluation  were  designed  and  commissioned  in  the  1950s  and 
’60s.  These  facilities  remain  the  backbone  of  the  aeronautical  development  process,  although  they 
are  becoming  more  challenging  to  maintain.  On  the  other  hand,  rapid  advances  in  computer 
hardware  and  software  offer  the  potential  to  dramatically  alter  the  design  and  development 
process  for  flight  systems  through  the  application  of  computational  science  and  engineering. 

However,  after  40  years  of  promises  to  eliminate  the  need  for  test  facilities,  advanced 
computational  science  and  engineering  have  still  not  diminished  significantly  the  need  for  test 
facilities  or  reduced  the  overall  cycle  time  for  development  of  flight  systems.  As  many  wind 
tunnel  test  hours  are  used  today  to  develop  a  flight  system  as  were  used  20  years  ago. 
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experience;  high-fidelity  modeling  and  simulation;  performance  prediction;  physics 
modeling  reliability;  test  facilities;  validation  &  verification. 


Now  that  I  have  your  attention,  the 
title  of  this  article  is  actually  the 
wrong  question!  The  proper  debate 
needs  to  be  centered  on  how  Com¬ 
putational  Science  and  Engineering 
(CSE)  and  wind  tunnel  testing  can  be  integrated  to 
reduce  the  overall  cycle  time  for  development  of  an 
aeronautical  system.  If  CSE  could  actually  eliminate 
the  use  of  wind  tunnels  in  system  development,  the  net 
gain  to  the  acquisition  program  would  be  fractions  of  a 
percentage  in  cost  savings.  On  the  other  hand  reducing 
the  overall  cycle  time  by  merging  CSE  and  wind 
tunnel  testing  could  reduce  total  development  cycle 
time  by  months  to  years,  resulting  in  billions  of  dollars 
of  savings.  This  article  is  focused  on  understanding  the 
challenges  to  using  computational  methods;  delineat¬ 
ing  changes  required  in  people,  processes,  and  tools  to 
make  CSE  more  effective;  and  creating  a  vision  for  an 
integrated  CSE/testing  approach  to  reduce  develop¬ 
ment  cycle  time. 

First,  for  clarity,  we  need  to  define  what  we  mean  by 
CSE  relative  to  the  ubiquitous  phrase  “Modeling  and 
Simulation  (M&S).”  For  this  article,  we  will  focus  our 
attention  on  high-fidelity,  physics-based  modeling  and 
simulation  as  opposed  to  engagement  or  theater 
wargaming  models.  The  former,  which  we  will  refer 


to  as  CSE,  is  more  directly  reflective  of  the 
aeronautical  design  process  and  the  way  test  facilities 
are  used  to  develop  systems.  The  latter,  which  we  will 
refer  to  as  M&S,  is  more  associated  with  doctrine, 
tactics  and  techniques,  and  training  for  the  Depart¬ 
ment  of  Defense  (DoD).  Also,  we  will  use  the 
terminology  CSE  to  connote  that  we  are  talking  about 
the  entire  spectrum  of  physics-based  modeling  such  as 
Computational  Fluid  Dynamics  (CFD),  Computa¬ 
tional  Structural  Mechanics/Computational  Structural 
Dynamics  (CSM/CSD),  computational  electromag¬ 
netics,  or  computer-aided  engineering.  Embodied  in 
these  modeling  activities  are  also  computational  heat 
transfer  and  chemical  kinetics. 

It  is  important  for  our  discussion  to  maintain  the 
distinction  between  M&S  and  CSE.  M&S  and  CSE 
are  both  computational  simulations,  but  within  the 
upper  levels  of  the  DoD,  M&S  is  ascribed  to  any  and 
all  computational  simulations.  In  reality,  the  DoD 
focuses  its  management  structure  and  funding  on 
wargaming  and  training  M&S.  Aside  from  the 
activities  of  the  Office  of  the  Secretary  of  Defense 
(OSD)  High  Performance  Computing  Modernization 
Office  (HPCMO),  CSE  has  largely  been  left  in  the 
hands  of  Science  and  Technology  (S&T),  test  and 
evaluation,  and  engineering  function  within  the 
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government  as  well  as  academia  and  industry.  Conse¬ 
quently  it  has  evolved  in  an  ad  hoc  fashion,  albeit  with 
great  success  in  select  areas. 

It  is  also  important  to  understand  the  difference  in 
the  use  of  CSE  in  support  of  S&T  and  engineering.  In 
the  former,  great  strides  are  being  made  in  the  use  of 
peta-flop  (101S  floating  point  operations  per  second) 
computing  in  research  approaching  the  molecular  level. 
“Designer”  materials  are  being  developed  by  using 
computer  simulations  to  manipulate  molecules  to 
achieve  desired  properties.  While  the  S&T  community 
has  been  the  principal  driver  for  advances  in  CSE,  it 
has  only  been  recently  that  the  engineering  community 
has  become  a  key  component  of  the  CSE  cognoscenti. 
In  the  past,  the  engineering  design  function  has  relied 
on  legacy  capabilities  and  relatively  small  computer 
systems.  What  has  become  clear  in  recent  years  is  that  a 
major  justification  for  developing  even  larger  scale 
computer  systems  is  tied  more  to  the  engineering 
process  than  to  the  S&T  community  (i.e.,  the 
economic  justification  is  based  on  the  output  of  the 
product  development  process  for  commercial  and 
military  systems).  The  integration  of  multidisciplinary 
simulations  and  design  optimization  of  aerospace 
vehicles  throughout  their  mission  profile  is  primarily 
an  engineering  function  that  can  be  enhanced  by  next 
generation  CSE  capabilities.  It  is  also  the  engineering 
application  of  CSE  that  directly  competes  with 
engineering  use  of  wind  tunnels  in  the  development 
process.  In  this  article  we  will  focus  on  engineering 
applications  of  CSE. 

A  celebration  of  success 

Before  we  launch  into  a  detailed  discussion  of  why 
advanced  modeling  cannot  replace  testing  in  the  near 
future,  we  need  to  understand  that  CSE  has  indeed 
had  a  profound  impact  on  aeronautical  system 
development.  CSE  is  aggressively  used  by  the  aero¬ 
nautical  system  design  community  to  apply  simpler 
engineering  models  to  reduce  the  design  space  to  a 
limited  set  of  design  variables  that  meet  requirements. 
These  select  configurations  are  modeled  with  much 
higher  fidelity  physics-based  models  to  develop 
prototype  designs.  Although  these  designs  are  not 
robust  enough  to  eliminate  the  need  for  wind  tunnel 
testing,  they  do  reduce  the  need  for  multiple 
experimental  configuration  studies  in  the  wind  tunnel. 
Ironically,  advanced  computer-based  control  systems  in 
modern  military  aircraft  actually  drive  a  need  for  even 
more  wind  tunnel  data  per  configuration  to  ensure 
accurate  control  variables  over  the  entire  flight 
envelope.  CSE  is  not  currently  capable  of  providing 
the  control  system  inputs  accurately  enough  to  reduce 
the  overall  wind  tunnel  testing  requirements. 


Once  the  development  phase  starts  into  the  wind 
tunnel  phase,  CSE  is  heavily  used  as  a  complementary 
tool  with  testing.  Highlights  from  the  application  of 
CSE  at  the  U.S.  Air  Force’s  Arnold  Engineering 
Development  Center  (AEDC)  in  support  of  wind 
tunnel  testing  are  summarized  in  Kraft  and  Matty 
(2005).  Typical  applications  include  pretest  planning  to 
ensure  optimization  of  the  test  facility  and  instrumen¬ 
tation;  support  to  real-time  data  analysis  and  decision 
making;  analysis  of  potential  wind  tunnel  effects  such  as 
support  or  wall  interference;  extrapolation  of  wind 
tunnel  data  to  flight  conditions;  support  to  flight  testing, 
particularly  weapon  separation;  and  support  to  opera¬ 
tional  flight  issues  that  occur  after  a  system  has  been 
developed.  Also,  CSE  is  an  excellent  tool  late  in  the 
development  cycle  for  evaluating  incremental  effects  as 
well  as  anomalies  that  show  up  during  flight  testing. 

To  appreciate  the  impact  of  CSE  on  the  aeronautical 
development  process,  a  history  of  applications  of  CSE 
in  support  of  wind  tunnel  testing  at  AEDC  is 
illustrated  in  Figure  1.  In  the  late  ’80s  and  early  ’90s 
serious  engineering  calculations  were  being  performed 
even  though  the  state  of  the  art  in  computer  capability 
was  barely  at  the  giga-flop  (109  floating  point 
operations  per  second)  level.  An  inviscid,  Euler 
solution  for  an  F-15E  aircraft  complete  with  stores, 
pylons,  pods,  etc.,  using  about  1  million  grid  points 
could  be  computed  in  less  than  8  Central  Processor 
Unit  (CPU)  hours,  making  it  a  useful  engineering  tool. 
The  simulation,  augmented  and  validated  by  wind 
tunnel  data,  was  sufficiently  accurate  to  predict  the 
release  of  weapons  from  the  conformal  fuel  tanks  on 
the  F-l  5E  well  enough  to  safely  guide  flight  testing 
(Kraft  1994). 

Using  the  1988  calculation  of  the  F-15E  as  a 
baseline,  subsequent  advances  in  CSE  enabled  more 
physics  (unsteadiness  and  viscous  effects)  as  well  as 
refined  grids  to  produce  more  accurate  engineering 
solutions  in  less  time.  In  Figure  1,  complexity  is 
defined  as  the  product  of  the  number  of  processors 
used  times  the  number  of  grid  points  times  the  output 
in  number  of  solutions  per  week.  Most  of  the  advances 
in  capability  demonstrated  in  Figure  1  are  attributable 
to  rapid  advances  in  hardware.  Today,  it  is  not 
uncommon  to  use  30-50  million  grid  points  for  a 
time-accurate,  unsteady,  viscous  simulation  of  a 
complete  aircraft  and  produce  a  solution  in  less  than 
8  CPU  hours.  These  advances  were  made  at  AEDC, 
which  has  aggressively  applied  CSE  to  engineering 
problems  for  over  20  years  but  has  relatively  modest 
computing  horsepower  (Kraft  and  Matty  2005).  Many 
of  the  solutions  illustrated  in  Figure  1  used  only  32 
processors,  but  today  several  hundred  processors  are 
available. 
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Increasingly  complex  computational  fluid  dynamics  simulations  enabled  by  computer  hardware  and  software  advances. 


Some  of  the  anecdotal  success  stories  demonstrated 
with  the  CSE  applications  at  AEDC  represented  in 
Figure  1  include  the  following: 

•  F-15E — first  demonstrated  use  of  CFD  to  modify 
wind  tunnel  and  flight  testing  resulting  in  safe 
carriage  and  release  as  well  as  improved  operations 
for  the  F-15E.  Post-integration  of  CSE  with  testing 
resulted  in  no  flight  incidents  during  certification  of 
numerous  weapons  for  carriage  and  release  from  the 
F-15E.  The  CSE  tools  developed  for  the  F-15E 
have  been  persistently  updated  and  applied  to  every 
DoD  platform  since,  with  immeasurable  reductions 
in  cost  and  improvements  in  safety. 

•  F-22 — an  integrated  CFD/wind  tunnel  testing 
approach  reduced  the  cost  of  models  by  over  $8 
million  and  provided  insight  to  support  an 
aggressive  flight-test  program  for  weapon  sepa¬ 
ration  clearance  over  the  entire  flight  envelope 
with  minimum  sorties. 

•  B-1B  (not  illustrated) — CFD  simulations  of  flare 
ejections  eliminated  a  major  part  of  a  flight-test 
program  at  a  cost  savings  of  $500,000  to  solve  an 
operational  problem. 

•  Minature  Air-Launched  Decoy  (MALD) — CFD 
design  of  fins  for  use  in  the  wind  tunnel  enabled 
high-quality  data  to  be  obtained  in  a  smaller,  more 
cost-effective  wind  tunnel. 

•  Joint  Strike  Fighter — CFD  analyses  enabled 
refinement  of  the  inlet  for  the  vertical  or  short 
take  off  and  landing  configuration  during  devel¬ 
opment,  eliminated  a  wind  tunnel  entry  for  store 
separation  simulation  resulting  in  approximately 


$1  million  savings,  and  augmented  stability  and 
control  analyses. 

There  are  numerous  additional  anecdotal  accounts 
of  how  CSE  has  improved  wind  tunnel  testing,  but  the 
real  challenge  is  to  develop  an  improved  overall 
methodology  for  the  use  of  CSE  with  testing  to 
dramatically  reduce  the  cycle  time  for  development  as 
well  as  improve  performance  of  the  systems.  This 
article  formulates  the  issues  keeping  CSE  from  having 
a  larger  impact  on  development  and  creates  a  vision  for 
innovative  ways  to  integrate  CSE  and  testing. 

Why  hasn’t  CSE  already  replaced  testing? 

The  “tastes  great,  less-filling”  debate  between  CSE 
and  wind  tunnels  has  been  ongoing  for  over  30  years). 
The  classic  American  Institute  of  Aeronautics  and 
Astronautics  (AIAA)  Dryden  Lecture  delivered  by 
Dean  Chapman  in  1979  was  the  first  serious  salvo  in 
the  debate  (Chapman  1979).  Chapman’s  visionary 
article  clearly  identified  the  rapid  growth  in  CFD 
hardware,  software,  and  modeling  capabilities  that 
could  transform  the  aerodynamic  design  process.  Many 
of  his  CFD  projections  have  been  exceeded  over  the 
last  30  years.  (He  was  forecasting  breakthroughs  only 
through  the  ’90s  and  did  not  extend  his  vision  to  the 
scale  of  CSE  today.)  On  the  other  hand,  the  average 
number  of  wind  tunnel  hours  used  in  development 
of  commercial  and  military  aircraft  continued  to  grow 
(Melanson  2008)  over  that  same  30-year  period  despite 
wind  tunnel  efficiency  increasing  by  at  least  a  factor  of 
four  (Kraft  and  Huber  2009).  At  the  same  time,  more 
and  more  DoD  programs  (and  some  commercial 
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programs)  overrun  their  original  cost  and  schedule 
estimates.  So  what  gives?  Obviously  CSE  has  not 
effectively  changed  the  aeronautical  development 
process  to  the  degree  envisioned  by  Chapman. 

Very  simply,  advances  in  computers  even  to  peta- 
flop  (101S  floating  point  operations  per  second) 
performance  and  beyond  are  necessary  but  not  sufficient 
to  transform  the  aeronautical  development  process.  It 
takes  a  holistic  advance  in  the  integration  of  people, 
processes,  and  tools  to  enable  the  kind  of  revolution 
people  have  envisioned  for  the  last  3  decades.  Even 
more  than  the  tools,  the  people  and  processes  need  to 
be  better  understood  and  integrated  with  the  advanced 
computer  hardware  and  software  to  increase  the 
effectiveness  of  CSE  in  the  aeronautical  development 
process.  In  the  next  section,  we  will  explore  challenges 
to  the  technologies,  intellectual  capital,  and  processes 
that  will  have  to  be  overcome  to  achieve  the  full 
promise  of  CSE  in  the  development  process. 

Technological  impediments 

Indeed,  large-scale  computing  power  is  at  our 
doorsteps.  Although  access  to  peta-flop  computers  is 
necessary  for  a  revolution  in  applying  CSE  to  aeronau¬ 
tical  system  development,  it  is  definitely  not  sufficient. 
Having  software  that  can  efficiently  and  effectively  use 
massive  parallel  computing  power,  having  robust 
algorithms  for  complex  and  multidisciplined  applica¬ 
tions,  improving  modeling  of  essential  physical  phe¬ 
nomena,  and  systematically  verifying  and  validating  that 
the  tools  will  work  robustly  in  the  engineering 
environment  are  equally  important.  In  this  section  we 
will  highlight  some  of  these  technical  challenges. 

Software  scalability 

The  trend  in  high-performance  computing  archi¬ 
tecture  is  toward  massive  parallel  processing  to 
upwards  of  100,000  CPUs  or  cores.  These  trends  are 
being  driven  by  the  rapidly  growing  cost  of  further 
increases  in  processor  clock  speed  and  the  emergence  of 
power  density  and  cooling  requirements  as  dominant 
considerations.  High-performance  computing  centers 
are  now  requiring  megawatts  of  power  for  operation. 
Although  peta-flop  computers  are  already  available  in 
select  federal  computing  centers  and  exa-flop  (1018 
flops)  machines  are  on  the  horizon,  the  legacy  CSE 
software  tools  routinely  applied  to  design  and  devel¬ 
opment  problems  have  not  been  scaled  to  maximize  the 
use  and  gain  the  efficiencies  afforded  by  clusters  with 
tens  of  thousands  of  processors.  Most  codes  have  been 
optimized  to  run  on  fewer  than  100  processors.  CFD 
solution  algorithms  tend  to  scale  reasonably  well,  but 
many  algorithms  have  topped  out  around  512  CPUs 
and  only  a  few  have  operated  with  a  few  thousand 


cores.  The  very  best  CFD  codes  scale  linearly  to  5,000 
cores,  which  is  three  orders  of  magnitude  smaller  than 
the  potential  million-core  machines  envisioned  for  the 
future.  Even  the  highest  performing  CFD  algorithms 
quickly  lose  ground  when  significant  Input/Output 
(I/O)  is  required  to  grab  and  store  solutions  every  few 
time  steps  for  a  graphical  representation  of  an  unsteady 
flow  solution.  Other  CSE  algorithms  do  not  scale  as 
well  as  CFD  codes.  Hence,  even  though  peta-flop 
machines  are  becoming  available,  current  software 
scalability  limitations  do  not  enable  the  solvers  to  use 
all  of  the  hardware  capability.  One  of  the  strategies  to 
offset  this  near-term  lack  of  scalability  is  to  use  a  large 
number  of  available  cores  to  simultaneously  solve  a 
number  of  parallel  cases.  This  strategy  will  be  useful  in 
either  rapidly  reducing  the  design  space  in  the  early 
phases  of  concept  development  or  in  building  a 
significant  CSE  data  base  in  later  stages  of  development. 

Complexity 

As  computer  systems  have  advanced,  so  has  the 
complexity  of  aeronautical  systems.  Over  the  last 
30  years,  expanded  flight  envelopes,  super-maneuver- 
ability,  super-cruise,  low  observables,  and  advances  in 
materials  technology  have  made  it  more  challenging  to 
model  the  physics  of  military  flight  systems.  As 
suggested  in  Figure  1,  all  of  the  advances  in  computer 
hardware  and  software  have  been  absorbed  in  increas¬ 
ing  the  fidelity  of  more  complex  systems. 

A  significant  challenge  to  developing  a  full  flight 
system  is  the  integration  of  the  major  subsystems  (i.e., 
airframe/propulsion  integration,  airframe/structure  in¬ 
tegration,  electromagnetic  interference,  control  sys¬ 
tems,  and  airframe/weapon  systems).  The  major 
defects  frequently  found  late  in  the  development  cycle 
for  a  flight  system  usually  occur  at  the  interface  of 
major  subsystems  (e.g.,  aerodynamically  induced  struc¬ 
tural  failures).  For  example,  on  average  for  military 
aircraft,  10  structural  flaws  are  found  in  the  flight-test 
phase  even  after  a  comprehensive  ground-test  cam¬ 
paign  and  massive  application  of  CSE.  The  fixes  for 
these  structural  flaws  can  range  from  simple  to 
significant,  costing  as  much  as  $1  billion  and  delaying 
a  program  by  a  year  or  more. 

Although  significant  advances  in  multidiscipline 
dynamic  simulations  for  maneuvering  vehicles  have 
been  made,  the  fidelity  of  current  capabilities  in  terms 
of  grid  resolution,  model  complexity,  and  interdisci¬ 
plinary  coupling  is  still  only  a  fraction  of  what  is 
needed  in  the  long  run. 

Performance  predictions  versus  reliability 

What  is  frequently  overlooked  by  the  CSE  commu¬ 
nity  desiring  to  replace  testing  is  that  test  facilities  are 
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used  to  predict  not  only  the  performance,  but  also  the 
operability,  reliability,  and  maintainability  of  an 
aeronautical  system.  The  majority  of  the  ad  hoc  success 
stories  in  applications  of  CSE  have  to  do  with 
performance  predictions  only.  CSE  is  not  capable,  for 
example,  of  simulating  the  aeromechanical  perfor¬ 
mance  of  a  turbine  engine  over  its  mission  life  to 
ensure  that  it  will  be  reliable  enough  to  field.  It  is  also 
not  capable  of  simulating  the  dynamic  stressing  of  its 
structure  to  assure  its  reliability  to  stay  “on  wing”  for 
hundreds  of  hours.  Similarly,  CSE  is  not  robust 
enough  to  decide  if  an  engine  can  be  restarted  at 
altitude  or  survive  a  bird  strike  or  debris  ingestion. 
Comparable  limitations  exist  for  modeling  dynamic 
fatigue  cycles  for  the  aircraft  structure.  Historical  data 
demonstrate  that,  on  average,  ten  structural  failures  are 
discovered  in  flight  even  after  numerous  computer 
simulations  have  been  performed.  Consequently,  in 
spite  of  advances  in  computer  horsepower  and 
applications  of  CSE,  test  facilities  will  be  essential  to 
assure  a  system  is  operable,  reliable,  and  maintainable. 

Physics  modeling 

The  list  of  physics  modeling  challenges  that  inhibit 
the  robust  application  of  CSE  is  legend.  The  classical 
problems  in  applying  CFD  include  turbulence  model¬ 
ing,  boundary  layer  transition,  and  flow  separation.  For 
relatively  benign  attached  or  mildly  separated  flow,  the 
use  of  Reynolds  Averaged  Navier  Stokes  (RANS) 
codes  with  the  addition  of  large  eddy  simulations  has 
advanced  to  a  very  good  engineering  capability  but  still 
has  enough  inaccuracy  to  preclude  total  reliance  on  the 
computed  results.  For  vortex  dominated  or  massively 
separated  flows  typical  of  advanced  tactical  aircraft  at 
the  corners  of  the  flight  envelope,  the  CSE  tools  are 
not  nearly  as  capable.  The  dynamics  of  separated  flow 
have  a  large  impact  on  structural  dynamics,  stability 
and  control,  as  well  as  control  surface  response. 

Turbulence  modeling  may  be  one  of  those  intrac¬ 
table  engineering  problems  that  cannot  be  solved  with 
higher  performance  computing.  Turbulence  modeling 
in  today’s  CFD  codes  is  a  semi-empirical  approxima¬ 
tion  of  the  physics  of  turbulence  to  support  practical 
calculations.  To  enhance  predictions  using  turbulence 
modeling  requires  decreasing  the  size  of  the  numerical 
grids.  To  double  the  resolution  of  a  three-dimensional 
flow  problem  requires  a  factor  of  16  increase  in 
computer  horsepower. 

Although  the  promise  of  the  revolution  in  computer 
hardware  will  enable  this,  the  scalability  of  the  software 
will  make  it  challenging  to  fully  utilize  for  realistic 
geometries.  The  step  beyond  turbulence  modeling 
enabled  by  high-performance  computing  is  Direct 
Navier  Stokes  (DNS)  simulations.  DNS  does  not  make 


approximations  to  the  equations  of  motion  but  does 
require  a  billion  plus  mesh  point  grids.  Although 
research  in  this  area  is  progressing  for  relatively  benign 
geometries,  it  will  be  decades  before  DNS  will  be 
useful  for  relevant  geometries  of  flight  systems. 

High-speed,  hypersonic  flight  bring  in  another 
range  of  physics  modeling  challenges.  At  hypersonic 
conditions,  additional  physical  phenomena  such  as  real 
gas  chemistry,  conjugate  heat  transfer,  wall  catalicity, 
shock/shock  interactions,  etc.,  create  significant  prob¬ 
lems  for  CSE.  Compounding  the  physical  modeling 
issue  is  the  dearth  of  qualified  experiments  and  test 
facilities  to  explore  the  physics  and  provide  sufficient 
high  quality  data  to  validate  and  verify  the  models.  For 
example,  to  fully  benchmark  hypersonic  boundary  layer 
transition  phenomena  would  require  experiments  that 
encompass  a  wide  range  of  Reynolds  numbers,  Mach 
numbers,  angles  of  attack,  bluntness,  favorable  and 
adverse  pressure  gradients,  roughness,  waviness,  wall 
temperatures,  cross-flow  phenomena,  surface  catalicity, 
and  a  range  of  gas  chemistries.  Not  to  be  overlooked  is 
the  requirement  for  advanced  flow  diagnostic  tools  that 
can  be  applied  in  the  high-temperature,  high-pressure 
hypersonic  flight  regime.  A  critical  review  of  CSE  and 
testing  for  hypersonics,  presented  in  Kraft  and 
Chapman  (1993),  suggests  an  incremental  approach 
to  CSE  and  testing  to  overcome  the  challenges  to  each. 

Validation  &  Verification  (V&V) 

The  aeronautics  community  has  given  itself  one 
huge  “head  fake.”  There  are  numerous  (and  growing) 
conference  articles  showing  “good”  comparisons  be¬ 
tween  CSE  solutions  and  select  experiments.  These 
comparisons  have  been  the  basis  for  many  marketing 
efforts  to  try  to  make  the  argument  that  CSE  can 
duplicate  test  facilities.  However,  an  accumulation  of 
anecdotal  comparisons  does  not  result  in  a  robust  tool. 
Tinoco  (2008)  probably  expressed  it  best: 

“CFD  validation  cannot  consist  of  the  compar¬ 
ison  of  the  results  of  one  code  to  those  of  one 
experiment.  Rather ,  it  is  the  agglomeration  of 
comparisons  at  multiple  conditions ,  code-to-code 
comparisons,  an  understanding  of  the  wind 
tunnel  corrections,  etc.,  that  leads  to  the 
understanding  of  the  CFD  uncertainty  and 
validation  of  its  use  as  an  engineering  tool. 
Examples  include  comparisons  of  predictive  CFD 
to  subsequently  acquired  test  data.  The  question  is 
not  can  CFD  give  a  great  answer  for  one  or  two 
test  cases,  but  can  the  CFD  “processes”  give  good 
answers  for  a  range  of  cases  when  run  by  a 
competent  engineer ?  This  is  what  validation  for 
an  intended  purpose  is  all  about.  ” 
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Figure  2.  Experience  trends  in  aerospace  systems  development — reduced  opportunity  for  development  of  intellectual  capital. 


The  recent  AIAA  drag  prediction  workshop  com¬ 
pared  results  from  a  number  of  state-of-the-art  codes 
applied  by  experienced  CSE  practitioners  to  the 
prediction  of  drag  on  transonic  transport  aircraft 
configurations  (Vassberg  et  al.  2008).  The  workshop 
provided  a  very  broad  view  of  the  state  of  the  art  of 
CFD  applications  within  the  industry,  much  more  so 
than  that  which  can  be  garnered  by  an  isolated  study. 
In  fact,  by  reviewing  in  isolation  any  one  of  the 
individual  data  blocks,  one  may  arrive  at  different 
conclusions  from  those  determined  from  the  complete 
data  set  comparison.  For  example,  a  typical  publication 
may  show  how  successful  a  CFD  solution  matches  test 
data.  By  combining  a  large  set  of  solutions  from  many 
sources  around  the  world,  this  workshop  clearly 
showed  that  there  remains  much  room  for  improve¬ 
ment. 

The  need  for  robust  V&V  also  underscores  the 
requirements  to  put  error  bars  on  the  computational 
results  as  well  as  the  experimental  results.  However, 
one  must  exercise  caution  in  doing  so.  A  CSE  solution, 
since  it  is  deterministic  for  a  given  computation,  will 
have  zero  precision  errors  but  could  have  excessive  bias 
errors  driven  by  grid  resolution,  time  steps,  numerical 
dissipation,  boundary  conditions,  and  physics  model¬ 
ing.  On  the  other  hand,  experimental  data  can  have 
both  precision  and  bias  errors.  Precision  errors  at  the 
95%  confidence  level  are  usually  well  documented  in 
the  experiment,  but  attention  needs  to  be  paid  to  bias 
errors  driven  by  geometric  modifications  of  a  scaled 
model,  Reynolds  number  scaling,  wall  interference, 
support  interference,  etc.  Experimental  validation  data 
for  CSE  V&V  needs  to  be  well  documented  for 
precision  and  bias  errors.  Furthermore,  comprehensive 
V&V  of  CSE  needs  clear  identification  of  all  boundary 
conditions,  which  will  require  off-body  flow  measure¬ 
ments  for  completeness  of  the  experimental  data  base. 


Experience  and  intellectual  capital 

Increasing  the  use  of  CSE  versus  testing  is  a  two- 
edged  sword  relative  to  the  technical  talent  involved  in 
aeronautical  system  development.  On  the  one  hand, 
visual  output  from  high-fidelity  models  provides 
unprecedented  insight  into  flow  features  that  cannot 
be  obtained  in  any  other  way.  Being  able  to  “see” 
streamlines  and  vortex  patterns  on  flow  over  a  vehicle 
brings  new  understanding  in  the  causative  relations 
between  aerodynamic  shapes  and  vehicle  performance. 
The  tools  also  allow  relatively  rapid  evaluation  of 
changes  to  the  design,  which  in  its  own  way  introduces 
more  insight.  On  the  other  hand,  having  a  generation 
of  engineers  experienced  only  in  the  “zeros  and  ones” 
of  advanced  modeling  has  the  downside  of  limiting  real 
understanding  of  the  physics  of  the  problem,  especially 
when  extending  into  realms  beyond  the  physical 
fidelity  of  the  model.  The  experiential  insight  gained 
from  physically  measuring  phenomena  is  important  in 
two  ways — it  provides  more  depth  in  understanding 
and  is  absolutely  essential  to  guide  development  of 
models  to  capture  the  physics.  There  seems  to  be  a 
circular  argument  that  we  can  better  model  the  physics 
than  the  experiments  when  the  models  are  only  as  good 
as  our  physical  understanding  gained  from  experi¬ 
ments.  If  we  no  longer  have  experimental  facilities, 
how  do  we  advance  the  physical  representation  in  the 
models? 

It  is  painfully  apparent  in  the  aerospace  industry  that 
there  has  been  a  significant  decline  in  the  experience 
base  of  aeronautical  designers  and  developers.  As 
shown  in  a  RAND  study,  the  experience  base  for 
post-World  War  II  engineers  was  approximately  6-12 
new  design  aircraft  per  career  (Drezner  1992).  The 
number  of  new  military  aircraft  program  starts  per 
decade  is  shown  in  Figure  2.  In  the  1950s  there  were 
60  aircraft  programs  in  various  stages  of  development. 
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In  contrast,  aerospace  engineers  starting  their  careers 
today  may  experience  only  one,  maybe  two  new  system 
designs  during  their  careers.  The  decadal  decline  of 
career  opportunities  in  other  aerospace  technical  areas 
is  also  shown  in  Figure  2.  For  most  aerospace  systems 
such  as  rocket  engines,  turbine  engines,  high-speed  X- 
vehicles,  and  ground-test  facilities,  engineers  today 
have  far  fewer  opportunities  to  hone  their  skills  than 
their  predecessors.  Anecdotal  evidence  has  linked  this 
trend  to  problems  experienced  in  many  recent 
aerospace  development  programs.  Counterarguments 
point  out  that  rapid  advances  in  design,  manufacturing, 
and  information  technologies  used  in  the  design  and 
development  process  of  today’s  new  design  aircraft 
have  compensated  for  some  or  all  of  the  declining 
experience  base. 

A  study  was  performed  at  the  Massachusetts 
Institute  of  Technology  (MIT)  to  understand  whether 
the  application  of  large-scale  computer  simulation  to 
the  design  process  would  offset  the  inexperience  of 
aircraft  designers  (Andrew  2001).  The  study  explored 
results  from  multiple  aircraft  programs  covering  4 
decades  from  the  ’60s  through  the  ’90s.  Aircraft  weight 
management  through  the  development  cycle  is  a 
critical  and  well-documented  parameter  that  can  be 
compared  from  program  to  program  and  decade  to 
decade.  In  the  study,  there  was  clear  evidence  that 
weight  management  degraded  from  decade  to  decade 
and  was  clearly  linked  to  the  level  of  experience  of  the 
designers.  Key  findings  from  the  study  included  the 
following: 

•  Strong  linkage  exists  between  experience  and 
performance. 

•  Seventies-era  design  efforts  outperformed  ’90s  era 
in  weight  management. 

•  Test  phase  is  an  important  downstream  indicator 
of  design  performance — test  personnel  under¬ 
stood  design  flaws  through  exposure  to  recurring 
problems 

•  Modern  design  tools  are  graphically  compelling, 
but  reduced  experimental  experience  led  to 
deficiencies. 

While  simulation  and  automation  of  the  design 
process  certainly  helped,  it  did  not  substitute  for  the 
intuition  and  inspiration  that  contributed  to  successful 
new  and  innovative  designs.  Also,  such  automation  was 
only  marginally  effective  when  dealing  with  new  and 
untried  technologies  because  the  basic  information 
needed  for  the  computational  algorithms  was  missing 
or  of  low  fidelity.  Furthermore,  it  should  be  clear  that 
one  cannot  really  assess  a  design  only  on  the  computer. 
One  has  to  build  the  prototype  and  test  it,  otherwise 
design  flaws  will  flow  downstream  into  manufacturing 


and  operations.  The  earlier  that  design  flaws  are 
discovered  through  prototype  testing  the  better. 

In  the  MIT  study,  some  negative  effect  was  found  to 
be  associated  with  today’s  computational  tools.  Not  so 
much  the  tools  themselves,  but  with  regard  to  the  tacit 
knowledge  derived  when  interacting  with  them. 
Today’s  tools  are  much  less  effective  at  developing 
the  tacit  knowledge  of  the  users.  Sophisticated 
simulation  models  of  all  types,  some  with  realistic 
graphic  presentations,  seem  to  command  a  greater  level 
of  creditability  than  they  deserve  in  many  cases.  In 
digging  for  a  root  cause  to  some  design  issues,  it  was 
clear  that  there  were  significant  shortcuts  taken  with 
respect  to  supporting  wind  tunnel  testing  and  model¬ 
ing  efforts  needed  to  develop  a  model  worthy  of  the 
level  of  confidence  with  which  it  was  being  applied. 

It  is  hard  to  envision  that  the  late  Richard  T. 
Whitcomb  of  the  National  Aeronautics  and  Space 
Administration  (NASA)  Langley  Research  Center 
would  have  made  his  breakthrough  contributions  to 
aerodynamics  if  the  only  tool  he  had  was  a  computer. 
His  insights  into  transonic  area  ruling,  supercritical 
airfoil  sections,  and  winglets  came  about  by  persistent 
experimental  research  and  sound  physical  understand¬ 
ing  of  the  flow  phenomena  (Hansen  1986).  The  three- 
legged  stool  of  theory,  experiments,  and  computations 
is  necessary  to  make  real  advances  in  the  aeronautical 
sciences. 

Processes 

CSE  is  just  a  single  tool  in  the  systems  engineering 
process  required  to  design,  develop,  and  field  an 
aeronautical  system.  Consequently,  if  the  CSE  com¬ 
munity  and  its  practitioners  are  not  equally  fluent  in 
understanding  the  overall  processes,  CSE  will  generally 
not  have  the  desired  effect  on  overall  development. 
Ensuring  the  process  environment  is  conducive  to 
integration  of  CSE  may  be  the  single  most  important 
consideration  for  advancing  CSE! 

Cultural  acceptance 

The  application  of  CFD  to  aeronautics  over  the  past 
40  years  has  seen  some  interesting  dynamics  in 
acceptance  by  the  community.  In  the  early  ’70s  when 
CFD  was  just  emerging  as  a  viable  tool  for  augmenting 
aeronautical  development,  the  “young  Turks”  engaged 
in  its  development  were  enthusiastic  about  its  poten¬ 
tial.  However,  the  managers  making  decisions  at  that 
time  had  not  grown  up  in  an  environment  of  CSE  and 
were  not  prone  to  support  a  large  scale  application  of 
CSE.  In  reality  the  tools  were  not  quite  mature  enough 
to  have  a  major  effect. 

After  a  generation  of  CSE  fledgling  applications, 
the  original  “young  Turks”  became  the  mid-level 
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decision  makers  in  the  ’90s  and  were  influential  in 
increasing  the  applications  of  CSE.  They  were  the 
zealous  advocates  for  CSE  being  able  to  replace  or 
reduce  the  need  for  traditional  development  tools  such 
as  testing.  However,  by  the  early  2000s,  it  was  painfully 
obvious  their  predictions  were  not  coming  true,  leading 
to  a  backlash  in  credibility  for  M&S. 

Why  didn’t  they  succeed?  Simply,  they  oversold  the 
capabilities  of  CSE.  In  their  enthusiasm  for  its  technical 
potential,  they  underestimated  the  people  and  process 
issues  discussed  in  this  article.  Also,  in  the  ’90s  the  DoD, 
recognizing  the  potential  ascendance  of  M&S,  decided  to 
organize  their  efforts.  The  DoD,  however,  largely 
organized  and  funded  M&S  for  wargaming  and  training, 
which  left  CSE  (as  we  use  the  terminology  in  this  article) 
to  ad  hoc  development  by  the  research  and  engineering 
communities.  Although  highly  successful  in  support  of 
wargaming  and  training,  M&S  is  not  appropriate  for 
modelling  physical  characteristics  of  material  systems  and 
hence  has  been  unsuccessful  in  eliminating  tests  or 
reducing  cycle  time.  Unfortunately,  the  limitations  of 
M&S  in  supplanting  the  need  for  testing  have  produced 
a  negative  inference  on  CSE  as  well. 

Another  cultural  dynamic  that  impedes  the  successful 
application  of  CSE  to  major  programs  is  the  lack  of 
understanding  that  one  needs  to  invest  in  a  capability 
before  taking  the  promised  gains.  It  is  not  uncommon  in 
the  DoD  to  take  the  forecasted  savings  from  M&S  up 
front  usually  by  diminishing  the  resources  for  testing. 
The  need  to  invest  in  and  implement  the  CSE  tools  to 
support  the  projected  savings  is  usually  not  budgeted.  As 
a  consequence,  the  modeling  and  the  testing  efforts  in 
support  of  system  development  both  come  up  short, 
leading  to  further  skepticism  about  M&S  in  general. 

Finally,  it  needs  to  be  recognized  that  the  aeronautical 
development  community  is  very  conservative.  Their 
design  and  development  processes  have  been  refined 
over  generations  of  applications  and  are  intended  to 
reduce  risks.  Coupled  with  the  forecast  for  fewer  major 
aeronautical  system  developments,  it  will  be  challenging 
to  have  the  industry  perform  a  significant  overhaul  to 
their  processes,  no  matter  how  attractive  CSE  appears. 
To  further  advance  CSE  into  the  development  process 
will  require  a  clear  advantage  to  the  program  manager 
relative  to  better  quality  of  data,  lower  costs,  reduced 
risk,  or  reduced  cycle  time.  Currently,  shifting  from 
testing  to  CSE  is  viewed  as  a  risk  without  clear  changes 
in  quality,  costs,  or  cycle  time. 

Concept  of  Operations  (CONOPS) 

Application  of  CSE  in  an  S&T  environment  to  a 
few  predictions  of  the  performance  of  a  vehicle  is 
woefully  short  of  the  operational  needs  required  for  the 
development  of  a  system.  The  current  operational 


model  for  large  scale  computers  in  the  DoD  and 
Department  of  Energy  are  suited  best  for  S&T.  To 
obtain  access  to  the  large  number  of  processors  needed 
to  supply  peta-flop  computing  for  S&T,  it  is  acceptable 
to  queue  up  a  number  of  very  large  batch  problems  and 
take  days,  weeks,  or  even  months  of  “wall  clock  time.” 
The  engineering  design  and  development  process  will 
require  a  significantly  different  CONOPS  to  succeed. 
In  the  early  phases  of  design,  literally  thousands  of 
configurations  need  to  be  evaluated  quickly,  albeit  with 
simpler  engineering  models.  As  the  design  matures,  a 
handful  of  parametric  factors  need  to  be  evaluated  with 
higher  fidelity.  As  the  design  matures  further,  the  data 
requirements  rise  exponentially.  More  data  are  re¬ 
quired  with  very  high  accuracy  on  shorter  cycles,  and 
large  databases  need  to  be  obtained  for  loads,  stability 
and  control,  subsystem  integration,  flight  simulation, 
etc.  Quick  turnaround  computing  to  support  interac¬ 
tive  design  is  essential.  This  intensity  of  schedule, 
accuracy,  and  volume  cannot  be  supported  by  compet¬ 
ing  in  a  queue  with  S&T  projects  (i.e.,  a  dedicated 
facility  will  be  required). 

The  dedicated  use  of  engineering  models  in  the  very 
early  phases  of  the  design  process  will  almost  certainly 
be  performed  on  proprietary  systems  within  the 
aerospace  industry.  However,  it  is  not  envisioned  that 
industry  will  invest  in  peta-scale  computing  resources 
even  though  the  unit  cost  of  computing  is  dropping 
dramatically.  Industry  chose  to  stay  at  modest  levels  of 
computing  capability  in  the  ’90s  and  to  rely  on 
government  investments  to  have  access  to  larger 
systems  (Mavriplis  et  al.  2007).  Consequently,  high 
fidelity  peta-flop  computer  systems  will  be  limited  to  a 
few  federal  sights  for  the  foreseeable  future.  This  could 
limit  their  application  to  developmental  engineering 
without  a  better  CONOPS.  National  capabilities  may 
need  to  be  scheduled  for  dedicated  applications  to 
major  systems  in  development,  much  as  government 
national  wind  tunnels  are  scheduled.  There  may  not  be 
enough  peta-scale  computing  capacity  at  the  national 
level  to  simultaneously  support  the  S&T  and  aeronau¬ 
tical  engineering  community  during  a  major  DoD 
development.  Also,  these  same  large-scale  computers 
will  need  to  support  other  government  acquisition 
programs  such  as  naval  ship  design.  Clearly,  a  strategy 
for  providing  sufficient  capacity  as  well  as  a  CONOPS 
to  support  design  and  development  of  systems  will  be 
required  to  enable  any  potential  success  for  large-scale 
application  of  CSE  to  the  development  process. 

Managing  the  process  requires  a  monopsony 

When  trying  to  understand  the  reasons  why  high 
fidelity  CSE  hasn’t  had  a  larger  impact  on  aeronautical 
system  development,  it  is  worthwhile  to  identify  the 
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common  attributes  of  those  areas  where  significant 
inroads  have  been  made.  It  is  the  author’s  observation 
that  CSE  has  had  a  significant  impact  on  aeronautical 
system  development  in  the  following  instances: 

•  The  process  is  controlled  by  a  single  organization 
that  can  ensure  the  use  of  CSE  in  design  and 
development. 

•  The  organization  has  a  substantial  and  sustained 
organic  capability  dedicated  to  building  and 
applying  CSE  tools  in  a  rigorous  development 
process. 

•  The  organization  has  at  least  de  facto  V&V  of 
their  tools  as  well  as  a  sustained  knowledge  base 
of  the  lessons  learned  from  the  application  of 
CSE  across  multiple  systems. 

Two  pockets  of  success  that  meet  these  criteria  stand 
out — design/development  of  commercial  aircraft  and 
the  certification  of  air  armament  on  military  aircraft. 
The  first  case  is  obvious — a  commercial  aircraft 
company  owns  the  entire  design  and  development 
process,  maintains  its  own  data  bases  and  tools  as  a 
competitive  edge,  and  sustains  a  critical  mass  of 
experienced  practitioners.  Since  the  CSE  tools  are 
used  consistently  from  program  to  program  internally, 
there  exists  within  the  aircraft  company  a  knowledge 
base  on  their  use  and  their  validity. 

In  the  second  case,  the  Air  Force  Seek  Eagle  process 
for  certifying  the  safe  carriage  and  release  of  air 
armament  is  the  primary  example.  The  AF  Seek  Eagle 
Office  (AFSEO)  owns  the  process  for  air  armament 
certification  recommendations.  Consequently,  AFSEO 
has  complete  control  of  the  use  of  modeling,  ground 
testing,  and  flight  testing  in  the  certification  process. 
In  conjunction  with  AEDC,  AFSEO  has  aggressively 
developed  and  applied  advanced  CFD  modeling  to 
simulate  the  carriage  and  release  of  weapons  from 
aircraft  for  over  20  years  (Kraft  1994;  Carlson,  King, 
and  Patterson  1995;  Benek  and  Kraft  1996;  Dean  et  al. 
2007).  The  advanced  CFD  tools  have  been  fully 
integrated  with  ground  and  flight  testing  to  provide  an 
effective  approach  to  weapon  separation  (Keen  et  al. 
2009).  The  community,  now  including  the  U.S.  Navy 
(Cenko  2009),  has  developed  a  common  set  of  tools,  a 
library  of  grid  models  for  important  DoD  aircraft  and 
air  armament,  and  a  body  of  knowledge  of  CSE 
applications  including  validation  and  verification.  This 
has  culminated  in  the  HPCMO-funded  Institute  for 
High  Performance  Computing  to  Air  Armament 
Applications  (IHAAA),  which  has  built  the  tools, 
refined  the  applications  process,  documented  a  com¬ 
mon  models  library,  and  created  a  critical  mass  of 
experts. 


In  the  general  development  of  military  aircraft,  there 
is  not  a  single  process  owner.  Although  the  Original 
Equipment  Manufacturers  (OEMs)  have  their  own 
internal  design  capabilities  used  to  support  develop¬ 
ment,  the  development  community  at  large  does  not 
have  an  integrated  set  of  CSE  tools.  The  OEM  tools 
and  databases  are  considered  proprietary;  hence  they 
cannot  be  used  by  the  broader  community,  particularly 
on  different  programs.  In  addition,  the  OEM  tools 
have  a  wide  range  of  levels  of  fidelity,  different 
providers  with  different  interface  standards,  a  lack  of 
rigor  of  recognized  V&V,  and  an  unwillingness  to 
compromise.  DoD  acquisition  policies  introduced  in 
the  1990s  relinquishing  total  system  performance 
responsibility  to  the  OEM  has  been  a  major  detriment 
to  fully  integrating  CSE  into  the  design  and 
development  processes  for  military  systems. 

Hence,  to  fully  implement  CSE  into  the  design  and 
development  of  military  flight  systems  will  require  the 
government  to  create  a  monopsony  (a  single  customer 
vice  a  single  supplier  as  in  a  monopoly).  The 
monopsony  for  design  and  development  of  flight 
systems  will  require 

•  government  guidance  on  the  systems  engineering 
approach  to  design  and  development  fully 
integrating  testing  and  CSE; 

•  a  common  architecture  for  applied  CSE  enabling 
optimization  for  large-scale  computing,  multidis¬ 
ciplinary  dynamic  simulations,  standard  libraries 
and  data  bases  for  DoD  systems; 

•  a  modular  “plug-and-play”  environment  permit¬ 
ting  OEMs  to  use  their  own  proprietary  CSE 
tools,  but  in  the  common  development  process; 
and 

•  a  critical  mass  of  government  CSE  applications 
experts  to  ensure  development  and  sustainment  of 
the  common  architecture  as  well  as  provide  the 
government  the  ability  to  perform  independent 
assessment  of  OEM  designs  during  the  acquisi¬ 
tion  process. 

Reengineering  the  aeronautical  system 
development  process  to  increase 
effectiveness 

So  we  now  come  full  circle.  The  proper  national 
debate  that  needs  to  be  held  is  not  CSE  versus  test 
facilities.  The  aeronautics  community  would  be  better 
served  putting  their  energy  into  creating  a  vision  for 
how  CSE  can  be  integrated  with  physical  testing 
processes  to  increase  the  effectiveness  of  both  during 
the  development  of  systems.  Effectiveness  in  the 
context  of  this  article  means  the  ability  to  reduce  the 
overall  cycle  time  for  development  while  minimizing 
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the  need  for  rework  of  late  defect  discoveries.  The 
elements  that  need  to  be  advanced  to  reengineer  the 
aeronautical  development  process  include  CSE  as  well 
as  test  facilities.  In  addition,  a  vision  needs  to  be 
created  for  innovative  ways  to  bring  CSE  and  testing 
together  to  have  the  maximum  impact  on  the 
effectiveness  of  the  development  process. 

The  CSE  tools  that  will  enable  a  monopsony  for 
aeronautical  development  are  being  developed  under 
the  OSD  HPCMO  Computational  Research  & 
Engineering  Acquisition  Tools  and  Environment 
(CREATE)  program.  CREATE  is  developing  ad¬ 
vanced  modeling  capabilities  to  support  aeronautical, 
naval,  and  radio  frequency  design.  CREATE-AV  (air 
vehicle)  is  the  aeronautical  program  under  CREATE 
and  is  focused  on  the  use  of  CSE  tools  across  the  entire 
spectrum  of  development  and  sustainment  of  aeronau¬ 
tical  systems  (Morton  et  al.  2009).  By  analyzing 
common  computational  needs  for  more  than  20 
acquisition  program  engineering  activities  from  con¬ 
cept  evaluation,  system  development,  through  imple¬ 
mentation  and  sustainment,  the  CREATE-AV  team 
has  been  able  to  determine  a  compact  set  of  advances 
required  in  CSE.  The  CREATE-AV  team  determined 
there  are  four  key  software  products  needed  by  the 
acquisition  engineering  workforce  that  fit  within  the 
available  budget  and  are  accomplishable  in  the 
CREATE  program  timeline.  The  four  software 
products  are  Helios,  a  virtual  helicopter  simulation 
tool;  Kestrel,  a  virtual  fixed-wing  aircraft  simulation 
tool;  Firebolt,  an  airframe-propulsion  integration 
simulation  tool;  and  Da  Vinci,  a  conceptual  design 
tool.  All  four  tools  are  currently  under  development. 

An  important  CREATE  software  design  philosophy 
that  will  support  use  by  the  community  is  modularity. 
A  common  architecture  in  CREATE-AV  is  a  Python- 
based  infrastructure  and  executive  and  either  C  or 
Fortran  90/95  components.  This  allows  a  build-up 
approach  to  adding  capability  and  multidisciplinary 
physics.  It  also  allows  a  factored  approach  to  the 
software,  aiding  in  code  maintenance  and  supportabil- 
ity.  This  approach  also  allows  all  of  CREATE  to  share 
components  among  software  products  to  reduce  the 
cost  of  development.  Particularly  noteworthy  is  an 
additional  executables  interface  that  would  permit  any 
proprietary  computational  module  used  by  the  OEMs 
to  stay  proprietary  within  their  application,  but  make 
the  output  available  to  the  government  evaluation  of 
the  system  performance. 

Implementing  new  technologies  to  maximize  effec¬ 
tiveness  will  require  changes  to  test  facilities  as  well. 
Furthermore,  older  facilities  will  eventually  reach  a 
point  where  they  become  too  costly  to  sustain  and 
upgrade,  and  building  new  is  more  cost-effective. 


However,  when  such  thresholds  are  reached,  these 
moments  become  opportunities  to  design  from  the 
outset  facilities  whose  functionality  reflects  compre¬ 
hensively  our  vision  for  how  to  conduct  aeronautical 
ground  testing.  Kraft  and  Huber  (2009)  have  created  a 
vision  for  what  future  aeronautical  ground  test  facilities 
need  to  look  like  to  support  better  integration  of  CSE 
and  to  increase  their  effectiveness.  Some  of  the 
attributes  required  for  upgrades  to  current  facilities  or 
for  future  test  facilities  include: 

•  ability  to  install  and  de-install  test  articles  in 
minutes  to  support  high-frequency,  short-dura¬ 
tion  tests  focused  in  areas  where  primary 
uncertainties  exist  and  to  optimize  use  of  Design 
Of  Experiments  (DOE); 

•  ability  to  rapidly  prototype  and  manufacture 
models  reflecting  design  changes  that  are  in¬ 
stantly  transmitted  by  customers  of  ground  test 
facilities  to  their  test  partners  using  the  latest  in 
compatible  CAD/CAM  and  model  shop  tools 
and  materials; 

•  ability  to  efficiently  modify  test  conditions  or 
proceed  through  a  test  point  matrix  to  minimize 
energy  usage  while  reflecting  to  a  maximum 
extent  DOE  considerations; 

•  convenient  and  thorough  optical  accessibility  for 
flow  diagnostics  tools; 

•  connectivity  to  high-performance  computing 
capabilities  to  integrate  and  merge  CSE  simula¬ 
tions  and  test  data; 

•  advances  in  data  mining  and  data  merging 
software  as  an  integral  part  of  the  facility  data 
systems  to  enable  rapid  analyses  of  the  variances 
along  response  surfaces;  and 

•  virtual  presence,  networking,  and  connectivity  to 
achieve  a  fully  integrated  Developmental  and 
Operational  Test  (DT/OT)  approach  in  an 
interoperable  environment. 

To  bring  CSE  and  test  facilities  into  a  unified 
toolset  for  streamlining  the  aeronautical  development 
process  requires  a  focus  on  effectiveness  of  the  process, 
not  just  the  efficiency  of  the  tools.  CSE  has  to  be  fully 
integrated  with  ground  and  flight  testing  to  reduce  the 
overall  cycle  time  for  development.  Kraft  (1995) 
introduced  a  holistic  approach  to  integrating  CSE 
with  testing  using  a  systems  approach.  Concepts 
evolved  from  the  application  of  CSE  to  weapons 
integration  led  to  a  broader  approach  for  acquisition 
programs  by  recognizing  CSE  as  the  potential  unifying 
backbone  for  system  knowledge  management  across 
the  development  cycle.  The  integrated  approach 
described  by  Kraft  (1995)  reinforces  the  need  to  have 
a  monopsony  for  managing  the  tools  and  knowledge 
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across  the  entire  development  process  to  impact 
acquisition  programs. 

A  primary  objective  measure  for  determining  the 
effectiveness  of  the  aeronautical  development  process  is 
acquisition  cycle  time.  Using  CSE  to  reduce  cycle  time 
will  have  a  greater  overall  influence  on  decreasing 
program  costs  and  justifying  CSE  applications  than 
any  other  cost-cutting  strategy.  Trying  to  justify  CSE 
only  as  an  offset  to  testing  misses  the  best  business 
case,  since  testing  is  only  a  small  fraction  of 
development  costs.  Reducing  cycle  time  for  major 
programs  that  can  expend  11-3  million  per  day  is  much 
more  cost-effective  than  reducing  testing.  Continued 
emphasis  on  the  efficiency  of  producing  data  has 
marginal  return  on  investment.  For  example,  the  cost 
of  a  wind  tunnel  campaign  for  development  of  a  twin 
engine  fighter  is  about  5  percent  of  the  overall  cost  of 
T&E.  In  turn,  the  total  cost  of  T&E  for  a  development 
program  is  generally  just  a  few  percent  of  the  total 
development  cost.  Hence,  a  50%  reduction  in  the  unit 
cost  of  a  wind  tunnel  campaign  equates  to  just  a  few 
tenths  of  a  percent  reduction  in  program  costs. 
Reducing  cycle  time  by  months  can  easily  save  a  major 
development  program  on  the  order  of  $1  billion. 

Cycle  time  can  be  estimated  by  the  following 
relationship: 

Workload 

Cycle  Time=  — — - ; — 

q-  Capacity 

In  this  expression,  Cycle  Time  is  the  total  time  required 
for  system  development.  Workload  is  the  total  amount 
of  work  to  be  accomplished  (e.g.,  man-hours,  test  unit 
occupancy  hours,  data  points,  computed  cases);  q  is  a 
quality  measure  that  indicates  the  fraction  of  the  total 
work  that  is  done  right  the  first  time  (i.e.,  the  inverse 
of  late  defects  and  rework);  and  Capacity  is  the  amount 
of  work  per  unit  of  time,  which  depends  on  the 
availability  of  the  development  infrastructure  (testing 
and  CSE),  the  staffing  to  use  the  capabilities,  and  the 
throughput.  The  three  primary  levers  to  decrease  cycle 
time  are  reducing  the  workload  required,  minimizing 
rework,  and  increasing  capacity. 

The  total  workload  involved  in  aeronautical  system 
development  is  primarily  process  driven.  For  example, 
if  a  wind  tunnel  campaign  for  a  major  fixed-wing 
aircraft  requires  about  22,000  hours  of  wind  tunnel 
testing,  then  given  today’s  national  capacity  of  about 
6,000  h/y,  such  a  campaign  requires  3  to  4  years  to 
conduct.  Surprisingly,  wind  tunnel  campaigns  are 
traditionally  designed  around  test  hours,  not  test 
points.  That  is  why  a  fourfold  increase  in  productivity 
generated  by  the  wind  tunnel  community  in  the  1990s 
had  essentially  no  impact  on  reducing  the  number  of 
wind  tunnel  hours  for  the  F-35  program  as  compared 


with  the  F-22  program  performed  a  decade  earlier 
(Kraft  and  Huber  2009).  Given  more  efficient 
throughput,  the  users  of  wind  tunnels  take  more  data, 
rather  than  reduce  test  hours.  Anecdotal  discussions 
with  several  aircraft  companies  over  the  years  strongly 
suggest  that  a  large  fraction  of  the  data  acquired  in  the 
wind  tunnel  is  not  used  but  is  retained  as  a  “security 
blanket”  in  case  an  anomaly  arises.  Reengineering  the 
way  wind  tunnel  data  are  obtained  and  used  has  the 
potential  to  be  a  major  driver  for  increasing  the 
effectiveness  of  ground  testing.  Although  CSE  has 
perennially  offered  the  ability  to  reduce  overall 
workload,  it  has  been  offered  as  a  replacement  for 
testing.  Currently,  CSE  as  a  direct  replacement  for 
testing  cannot  come  anywhere  near  efficiently  replac¬ 
ing  the  total  wind  tunnel  and  flight  test  hours. 

Similarly,  the  inverse  of  q,  the  amount  of  rework 
normally  performed,  is  also  process  driven.  For  most 
aerospace  systems  in  development,  q  is  approximately 
0.25,  resulting  in  four  to  10  rework  cycles.  The 
incremental  increase  in  program  costs  is  proportional 
to(\/q)  —  1,  indicating  the  potential  to  easily  double 
development  costs  through  late  defects  and  rework. 
The  best  way  to  minimize  the  impact  of  rework  on 
cycle  time  is  early  discovery  of  defects.  This  will  entail 
improvements  in  design  methodologies  employed  by 
aircraft  companies  coupled  with  improvements  in  wind 
tunnel  testing  and  modeling  techniques.  These  latter 
improvements  minimize  any  defects  in  design  being 
passed  downstream  to  flight  testing,  where  the  cost  of 
fixing  the  defect  increases  an  order  of  magnitude.  Also, 
feedback  loops  from  discrepancies  found  in  flight 
testing  back  to  ground  testing  and  back  to  design 
methodology  need  to  be  institutionalized  to  make 
further  improvements.  A  primary  target  for  decreasing 
rework  is  improving  the  early  determination  of  the 
impact  of  steady  and  unsteady  flow  effects  on  the 
vehicle  structure.  Historically,  most  aircraft  develop¬ 
ment  programs  have  discovered  10  structural  flaws  in 
flight  with  varying  degrees  of  cost  and  schedule 
impacts  that  can  reach  a  billion  dollars  and  a  year  to 
overcome.  As  can  be  seen  from  this  example,  increasing 
q  (decreasing  late  discoveries)  will  have  a  profound 
effect  on  development  cycle  time  and  cost.  The  early 
reduction  of  defects  may  be  the  single  most  important 
area  for  the  use  of  CSE.  However,  multidisciplined 
approaches  will  have  to  be  improved  to  realize  the 
potential  gains  in  defect  reduction. 

In  contrast  to  process-driven  parameters,  capacity  is 
primarily  budget  driven.  Capacity  equals  the  availabil¬ 
ity  of  the  capability  times  the  staffing  available  to  use 
the  capability  times  the  throughput.  For  testing,  the 
availability  of  the  equipment  depends  on  investments 
in  maintenance  and  reliability.  Also,  the  budget 
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determines  whether  a  facility  is  staffed  for  one,  two,  or 
three  shifts.  Staffing  is  the  most  dynamic  variable  for 
increasing  or  decreasing  capacity.  Throughput  (e.g., 
test  points  per  hour,  solutions  per  day)  is  also  budget 
driven.  The  capacity  of  CSE  is  also  budget  driven.  The 
availability  of  large-scale  computers,  the  critical  mass 
of  intellectual  capital  to  use  the  capability,  and  the 
throughput  of  the  computations  will  similarly  drive 
cycle  time.  Developing  and  funding  integrated  test 
facilities  and  CSE  with  capability  and  capacity 
optimized  to  maximize  throughput  using  the  reduced 
workload  and  defect  avoidance  and  discovery  ap¬ 
proaches  will  be  a  powerful  adjunct  to  process 
reengineering. 

The  discussion  on  cycle  time  focuses  on  the  cycle 
time  for  testing.  To  aggressively  attack  the  cycle  time 
for  development  of  a  new  flight  system,  one  also  needs 
to  address  the  contributions  to  cycle  time  from  design, 
prototyping,  analysis  of  results,  and  other  development 
and  manufacturing  maturation  activities.  There  is 
potential  interplay  between  these  processes  and  those 
from  testing  that  can  further  help  reduce  overall  cycle 
time.  In  this  article  we  are  focused  on  reducing  the 
equivalent  cycle  time  for  testing  through  better 
integration  of  CSE. 

CSE  does,  however,  offer  significant  potential  to 
impact  the  overall  wind  tunnel  campaign  in  three 
significant  areas.  First,  and  most  importantly,  CSE  can 
be  used  to  reduce  the  overall  workload.  Second,  CSE, 
if  applied  appropriately,  can  reduce  downstream  effects 
of  late  defect  discovery  on  total  development  cycle 
time.  Third,  CSE  can  be  used  to  integrate  major 
subsystems  earlier  in  the  development  cycle  avoiding 
late  integration  issues. 

Minimizing  workload 

The  primary  target  for  reengineering  aeronautical 
development  to  increase  effectiveness  is  to  reduce  the 
overall  workload  without  increasing  risk.  A  major 
contributor  to  the  number  of  wind  tunnel  test  hours 
used  is  the  need  to  generate  about  2.5  million  data 
points  to  determine  the  stability  and  control  (S&C)  of 
the  vehicle.  This  is  traditionally  done  in  the  one  factor 
at  a  time  (OFAT)  mode  where  data  are  obtained  for 
each  model  configuration,  orientation,  speed,  and 
simulated  altitude  over  the  entire  operating  envelope. 
This  ponderous  number  of  data  points  also  has  been 
the  primary  reason  that  CFD  has  not  made  greater 
inroads  into  developmental  wind  tunnel  testing. 
Estimates  to  compute  the  equivalent  2.5  million 
OFAT  points  range  from  approximately  100  to 
1,200  years  using  existing  computer  tools. 

Recently,  the  CFD  community  introduced  an 
innovative  and  efficient  computational  method  for 


accurately  determining  the  static  and  dynamic  S&C 
characteristics  of  high-performance  aircraft  (Dean  et 
al.  2008).  In  contrast  to  the  “brute  force”  approach  to 
filling  an  entire  S&C  database  for  an  aircraft,  an 
alternate  approach  is  to  reduce  the  number  of 
simulations  required  to  generate  a  complete  aerody¬ 
namic  model  of  a  particular  vehicle  configuration  at 
selected  flight  conditions  by  using  one  or  a  few 
complex  dynamic  motions  (e.g.,  varying  frequency 
and  amplitude  over  a  dynamic  trajectory)  and  nonlinear 
system  identification  techniques.  This  approach  now 
makes  CFD  a  reasonable  source  of  S&C  data  for  an 
aircraft. 

Of  interest,  there  is  a  comparable  experimental 
technique  using  the  pre-filtered  dynamic  output  from 
the  force/moment  balance  used  in  the  wind  tunnel, 
system  identification  techniques,  and  a  “fly  the 
mission”  profile  in  the  wind  tunnel.  Recent  advances 
have  been  made  in  demonstrating  control  systems  that 
permit  a  wind  tunnel  to  respond  in  real  time  to 
changing  Mach  number  and  pressure  altitude  while 
maneuvering  the  test  article  to  fly  the  mission  versus 
building  the  massive  data  base  using  OFAT  methods 
(Sheeley,  Sells,  and  Felderman  2010). 

As  indicated  in  Figure  3,  using  these  advanced  “fly 
the  mission”  modeling  and  testing  methodologies 
combined  with  design  of  experiments  offers  an 
innovative,  aggressive  approach  to  reducing  the  overall 
test  workload.  Attempts  to  apply  DOE  to  streamline  a 
traditional  individual  wind  tunnel  test  have  been  only 
marginally  successful  because  current  wind  tunnels  are 
not  conducive  to  rapidly  changing  parameters  to 
optimize  randomness  of  the  data  set.  However,  if  one 
shifts  to  thinking  about  DOE  at  the  “campaign”  level 
there  may  be  a  more  productive  approach  to  using 

DOE. 

Instead  of  the  OFAT  approach  to  building  the 
colossal  data  base  characteristic  of  today’s  aeronautical 
development  processes,  an  approach  using  DOE 
response  surface  techniques  could  be  more  effective. 
A  response  surface  is  a  mathematical  construct  that 
represents  the  parameter  space  along  which  the 
characteristics  of  the  vehicle  are  captured.  An  example 
of  the  use  of  response  surface  modeling  for  aerody¬ 
namic  configurations  is  given  in  Landman  et  al.  2007. 

In  contrast  to  traditional  OFAT  approaches  that 
basically  fill  up  the  entire  parameter  space  and  try  to 
interpolate  to  determine  the  characteristics  of  the 
vehicle,  an  initial  response  surface  could  be  built  using 
simple  engineering  models.  Of  course  the  uncertainty 
over  the  response  surface  would  be  high,  but  more 
refined  high-fidelity  physics  modeling  could  then  be 
efficiently  applied  to  reduce  the  uncertainties  over  the 
response  surface  using  the  fly  the  mission  approach 
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Figure  3.  Streamlining  the  aeronautical  development  process  by  merging  modeling  and  testing  using  design  of  experiments. 


mentioned  above.  Those  areas  on  the  response  surface 
that  still  exhibit  a  high  degree  of  uncertainty  then 
become  the  primary  focus  for  the  wind  tunnel  test 
campaign  (i.e.,  the  focus  is  put  on  key  areas  for  risk 
reduction  versus  defining  the  entire  parameter  space). 
DOE  coupled  with  estimation  theory  could  help 
determine  the  minimum  number  of  computations  or 
test  points  to  reduce  uncertainties  in  areas  of  interest  on 
the  response  surface.  Finally,  the  areas  of  residual 
uncertainty  become  the  primary  interest  for  focused 
flight  testing,  which  serves  to  reduce  the  overall 
workload  for  that  phase  of  testing.  In  this  manner,  the 
overall  amount  of  testing  could  be  dramatically  reduced 
with  a  commensurate  effect  on  total  cycle  time. 

The  integration  of  multidisciplinary  data  is  key  to 
developing  this  aggressive  approach  to  minimizing  data 
requirements.  Multidimensional  meta-models  can  be 
automatically  constructed  using  limited  experimental 
or  numerical  data,  including  data  from  heterogeneous 
sources  such  as  CSE,  ground  test,  and  flight  test. 
Recent  progress  in  multidimensional  response  surface 
technology  provides  the  ability  to  interpolate  between 
sparse  data  points  in  a  multidimensional  parameter 
space.  These  analytical  representations  act  as  surrogates 
that  are  based  on  and  complement  higher  fidelity 
models  and/or  experiments,  and  can  include  technical 
data  from  multiple  fidelity  levels  and  multiple 
disciplines  (Riesenthaal  et  al.  2006). 

The  mathematics  of  the  DOE  methodology  helps 
ensure  the  optimum  data  set  is  taken.  The  alpha  and 
beta  (or  power  coefficients)  of  the  DOE  process  can  be 
used  to  address  how  much  further  variance  can  be 
reduced  on  the  response  surface  by  an  additional 


calculation,  wind  tunnel  test,  or  flight  test.  There  is  a 
point  at  which  doing  another  CFD  solution  will  not 
reduce  uncertainty  further;  hence,  one  needs  to  move 
on  to  wind  tunnel  testing.  Likewise,  there  is  a  point  of 
diminishing  returns  for  doing  another  wind  tunnel  test, 
and  the  program  needs  to  move  on  to  flight  testing. 
Thus,  unnecessary  modeling  and/or  testing  can  be 
minimized.  Estimation  theory  (Deyst  2002)  can  be 
used  to  estimate  the  unit  cost  of  further  reductions  in 
uncertainty  leading  to  an  optimum  strategy  for 
combining  testing  and  modeling.  The  DOE  beta 
coefficient  also  provides  some  insight  into  the 
probability  that  a  defect  is  being  passed  downstream 
to  the  next  development  step. 

The  response  surface  method  also  provides  an 
invaluable  approach  to  supporting  integrated  develop¬ 
mental  testing  (DT)  and  operational  testing  (OT)  as 
well  as  addressing  networking  and  interoperability 
issues.  The  characteristics  of  the  vehicle  captured  in  the 
response  surface  can  be  translated  directly  into  the 
performance  math  engine  for  a  manned  flight  simu¬ 
lator  as  suggested  in  Figure  3.  Even  at  the  earliest 
phases  of  development,  a  manned  flight  simulator  can 
start  to  address  some  of  the  operational  integration 
issues,  thereby  allowing  integrated  DT/OT  earlier  in 
the  program.  If  early  brass-board  or  digital  models  of 
the  avionics  and  communications  packages  are  brought 
into  the  manned  flight  simulator,  the  evolving 
performance  of  the  system  can  be  evaluated  as  a  node 
in  a  distributed  mission  simulation.  Feedback  from  this 
integrated  approach  can  be  used  in  the  very  early  stages 
to  improve  the  design  for  maximum  performance  as  an 
interoperable  system.  Today,  most  of  the  OT  interface 
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Figure  4.  Root  cause  analysis  to  avoid  late  defect  discovery. 


issues  as  well  as  interoperability  are  not  addressed  until 
very  late  in  the  development  process.  The  overall 
impact  on  reducing  development  cycle  time  using  such 
an  innovative  approach  could  be  immense. 

Decreasing  late  defects 

Defects  discovered  late  during  the  development 
process  not  only  increase  cycle  time  but  also  can  impact 
manufacturing  costs  if  significant  tooling  and  produc¬ 
tion  have  already  occurred.  Since  concurrent  engineer¬ 
ing  is  routinely  used  to  reduce  procurement  cycle  time, 
almost  always  tooling  and  initial  production  are  in 
progress  by  the  time  flight  testing  occurs. 

The  challenge  to  reducing  late  defect  discovery  is  to 
determine  the  root  cause  for  reoccurring  late  defects.  A 
prime  example  for  the  need  to  better  understand  the 
root  cause  for  late  defects  is  the  frequency  of  structural 
failures  discovered  during  flight  testing,  even  after 
numerous  hours  of  analysis  and  wind  tunnel  testing 
were  used  to  design  the  aircraft  structure.  On  average, 
10  structural  failures  are  uncovered  during  flight 
testing  irrespective  of  the  type  of  aircraft.  In  addition, 
many  flight  systems  resize  control  surfaces  after 
discovering  inadequate  control  authority  during  flight 
testing.  Working  control  surface  sizing  and  structural 
issues  this  late  in  development  can  lead  to  significant 
delays  in  completion  and  considerable  cost  increases. 
Frequently,  the  late  defects  in  structure  or  control 
surface  size  are  looked  upon  as  unique  circumstances 
for  the  current  vehicle  in  testing.  However,  by 
evaluating  multiple  systems,  it  is  clear  that  there  may 
be  more  systemic  causes  for  these  late  defects. 


A  suggested  approach  to  combining  modeling  and 
testing  to  reduce  systemic  late  defects  is  illustrated  in 
Figure  4.  Using  Bayesian  statistics,  the  probability  of 
finding  a  structural  flaw  in  flight  is  an  accumulation  of 
the  probabilities  of  a  flaw  being  overlooked  either  in 
design,  analysis,  ground  testing,  or  assembly  of  the 
prototype  flight  article.  Since  the  flight  test  occurs 
several  years  after  the  design  and  ground  testing 
phases,  a  root  cause  analysis  of  structural  failures 
traceable  back  to  the  design,  analysis,  or  wind  tunnel 
testing  phase  is  essentially  never  done.  Consequently, 
these  systemic  issues  show  up  in  program  after 
program. 

The  first  step  in  reducing  the  discovery  of  late 
structural  defects  is  to  identify  the  reoccurring 
structural  problems  across  multiple  programs.  (This 
reinforces  the  need  for  a  monopsony  approach  by  the 
government  to  establish  a  knowledge  base  of  late 
defects  and  root  causes.)  Second,  the  systemic  issues 
need  to  be  traced  back  to  the  source  of  the  defect  (i.e., 
the  design,  the  analysis,  or  ground  testing).  CSE  can 
be  a  major  enabler  for  helping  to  assess  the  potential 
root  causes. 

Multidiscipline,  high-fidelity  CFD/CSD  can  be 
used  earlier  in  the  design  cycle  to  examine  interactions 
between  the  airframe  and  structure  Traditionally, 
pressure  loads  data  were  obtained  on  a  very  early 
(and  expensive)  wind  tunnel  model  specifically  de¬ 
signed  with  hundreds  to  a  few  thousand  pressure  taps 
on  the  surface  of  the  model.  These  pressure  loads  were 
provided  to  the  structural  engineers  to  perform  a 
structural  analysis  and  design  of  the  vehicle.  While  the 
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structural  engineers  are  doing  their  analyses,  the 
aerodynamicists  are  usually  continuing  to  refine  the 
outer  mold  lines  of  the  vehicle  to  improve  perfor¬ 
mance.  Because  of  the  cost  and  complexity  of  wind 
tunnel  pressure  models,  effects  on  pressure  loads  due  to 
changes  in  outer  mold  lines  were  usually  not  updated. 
When  the  airframe  and  underlying  structure  were 
integrated  into  the  first  set  of  flight  vehicles,  it  was  not 
uncommon  to  find  structural  flaws.  Contributing  to 
these  late  discoveries  are  inadequate  characterization  of 
the  dynamic  interactions  between  fluids  and  structures 
as  well  as  a  lack  of  integration  of  aerodynamic  and 
structural  analysis  tools. 

Modern  wind  tunnel  testing  implements  the  use  of 
Pressure-Sensitive  Paint  (PSP)  instead  of  physical 
pressure  taps  (Sellers  2005).  PSP  offers  the  opportunity 
to  provide  cost-effective  updated  structural  load 
information  as  the  aerodynamic  shape  of  the  vehicle 
changes.  Coupled  with  an  integrated  CFD/CSD 
modeling  of  the  airframe/structure,  it  will  be  possible 
to  better  define  the  static  and  dynamic  structural  loads 
prior  to  the  first  flight.  This  dynamic  interaction 
between  modeling  and  wind  tunnel  testing  can  be 
incorporated  into  Bayes’  equation  as  an  iterative 
learning  tool  to  reduce  uncertainties  in  the  results 
from  the  analysis  or  wind  tunnel  test  as  suggested  in 
Figure  4.  Using  Bayes’  equation  in  an  iterative  fashion 
between  CSE  and  wind  tunnel  testing  should  mini¬ 
mize  the  probability  of  uncertainties  being  passed 
downstream  in  the  development  process.  In  addition, 
DOE  power  coefficients,  if  properly  applied,  should  be 
able  to  quantify  the  probability  of  a  defect  being  passed 
downstream  to  flight. 

Bayesian  statistics  can  also  be  used  to  help  a  program 
manager  better  assess  the  risk  to  the  program  of 
permitting  known  design  issues  to  be  unresolved  until 
later  in  the  development  cycle.  The  trade  space 
between  cost,  schedule,  and  the  potential  impact  of  a 
late  defect  can  be  assessed  using  Bayesian  statistics  to 
define  a  value  proposition  relative  to  the  design  cycle. 

Finally,  CSE  can  be  an  invaluable  tool  to  ensure 
better  use  of  ground  test  facilities  to  preclude  design 
defects  from  finding  their  way  into  the  flight  test 
program.  Use  of  CSE  to  account  for  Reynolds  number 
scaling  effects  and  potential  bias  errors  such  as  wind 
tunnel  wall  interference  is  well  understood  and 
effectively  applied.  An  area  where  scaling  effects  are 
not  well  understood  and  CSE  may  have  the  potential 
for  producing  new  insights  is  simulation  of  military 
tactical  aircraft  at  high-angle  maneuvering  conditions. 
In  these  conditions,  the  flow  is  dominated  by  vortex 
structures  and  flow  separation.  Surprisingly,  a  large 
number  of  tactical  military  aircraft  have  required 
significant  modification  to  control  surface  size  or 


structure  even  after  a  comprehensive  wind  tunnel 
campaign.  Changes  of  this  magnitude  during  the  flight 
test  program  can  have  a  profound  effect  on  program 
cost  and  schedule.  Coupled  effects  on  manufacturing 
costs  can  also  become  significant  during  this  phase. 

There  exists  a  strong  potential  that  a  root  cause  for 
these  late  defect  discoveries  may  be  the  lack  of 
understanding  of  scaling  principles  for  vortex-domi¬ 
nated  or  separated-flow  phenomena.  The  general 
Reynolds  number  scaling  principles  used  today  were 
developed  in  the  mid  ’70s  from  attached  flow  data 
taken  on  commercial  transport  aircraft  configurations. 
At  the  time,  computational  tools  as  well  as  flow 
diagnostics  were  not  capable  of  supporting  more  in- 
depth  understanding  of  separated-flow  phenomena. 

At  high  angles  of  attack,  flow  separation  from  the 
leading  edge  can  create  vortex  structures  that  impinge 
on  vertical  tails.  The  appearance  and  interaction  of 
these  vortices  with  the  vehicle  can  strongly  influence 
control  authority  or  cause  structural  failures.  The 
classical  wing-drop  roll-control  problem  for  the  F-18 
was  caused  by  vortex-shock  interactions.  Vertical  tail 
structural  flaws  caused  by  vortex  impingement  and 
breakdown  have  been  discovered  on  a  number  of  twin- 
tail  flight  vehicles,  including  the  F-22. 

In  the  wind  tunnel,  the  model  is  generally 
geometrically  scaled.  If  one  examines  the  leading  edge 
vortex  formation  and  separation  for  a  typical  tactical 
fighter  at  high  angle  of  attack  there  are  at  least  five 
characteristic  lengths  involved  in  the  problem:  chord 
length,  leading  edge  radius,  boundary  layer  displace¬ 
ment  thickness,  vortex  core  diameter,  and  vortex 
breakdown  length.  It  is  not  clear  whether  these  are 
dependent-  or  independent-length  scales,  which  begs 
the  question  of  whether  geometric  scaling  is  sufficient 
to  model  vortex-dominated  or  massively  separated- 
flow  phenomena.  Current  CSE  tools,  including  large 
eddy  simulations,  have  been  used  to  model  vortex 
effects  on  aircraft  at  high  angles  of  attack  (Morton 
2009).  Coupled  with  advanced  off-body  flow  laser 
diagnostic  tools  like  Planar  Laser-Induced  Fluores¬ 
cence  (PLIV)  (Ruyten  1994)  CSE  could  provide  an 
integrated  computational/experimental  approach  to 
understanding  better  the  causative  effects  of  the  various 
length  scales  and  better  predict  flight  conditions  from 
wind  tunnel  data. 

Early  subsystem  integration 

Another  key  to  increasing  the  quality,  q,  or 
decreasing  the  amount  of  rework,  is  earlier  and  better 
integration  of  major  subsystems  such  as  the  airframe/ 
structure,  the  airframe/propulsion  systems,  or  the 
airframe/weapon  systems.  Most  defects  occur  at  the 
interface  of  major  subsystems.  Current  practices 
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generally  address  system  integration  issues  later  in  the 
development  process,  which  maximizes  the  amount  of 
rework  required  (and  increases  associated  costs)  if  a 
defect  is  discovered.  Key  enablers  required  to  get  earlier 
insights  into  integration  issues  include  high-fidelity, 
multidisciplined  modeling  capabilities.  These  multi¬ 
discipline  tools  are  being  developed  in  the  CREATE- 
AV  program  described  earlier. 

Integration  of  CSE  with  testing  for  airframe/ 
weapon  integration  is  already  a  mature  capability.  As 
mentioned  earlier  the  AFSEO  has  a  monopsony  on  the 
air  armament  certification  process.  As  a  consequence, 
CSE,  wind  tunnel  testing,  and  flight  testing  have  been 
highly  integrated  since  1988  (Kraft  1994;  Carlson, 
King,  and  Patterson  1995;  Dean  et  al.  2007;  Keen  et  al. 
2009;  Cenko  2009;  and  Kraft  1995).  However,  the 
Seek  Eagle  certification  process  occurs  after  an  air 
vehicle  is  developed,  so  the  tools  are  used  to  identify 
issues  and  avoid  parts  of  the  flight  envelope  where  the 
weapon  and  the  airframe  may  not  be  optimally 
integrated. 

The  tools  and  capabilities  developed  with  the 
government  to  support  airframe/weapon  integration 
have  migrated  to  use  by  industry  as  well.  However, 
these  advanced  modeling  tools  are  not  used  as  an 
integral  part  of  the  early  design  cycle  for  a  new  flight 
system.  This  is  partially  driven  by  the  fact  that  the 
entire  inventory  of  air  armament  to  be  carried  by  a  new 
platform  is  not  necessarily  defined  during  the  system 
development  phase.  However,  basic  inventory  weapons 
should  be  integrated  into  the  earliest  design  phases  to 
ensure  compatibility  downstream.  This  would  decrease 
the  probability  of  finding  interface  issues  between  the 
airframe  and  weapon  systems  much  later  in  the 
development  cycle. 

Airframe/structure  integration  is  arguably  the  most 
important  of  the  integration  issues  that  need  to  be 
resolved  early  in  the  design  cycle.  The  static  and 
dynamic  interactions  between  aerodynamic  flow 
around  the  vehicle  and  structural  integrity  of  the 
system  are  a  major  driver  in  weight  management  for 
the  vehicle  as  well  as  sizing  of  control  surfaces.  Weight 
management  over  the  development  cycle  is  a  major 
causative  factor  for  rework  and  cost  escalation.  Many  of 
the  key  performance  parameters  guiding  development 
of  the  system  are  affected  by  vehicle  weight.  Resolving 
weight  issues  late  in  the  development  cycle  also  can 
impact  determination  of  the  Reliability,  Availability, 
and  Maintainability  (RAM)  of  the  system  prior  to 
fielding.  RAM  is  one  of  major  causes  for  a  system  to  be 
determined  to  be  unsuitable  for  fielding. 

The  application  of  peta-scale  computing  in  the  near 
future  will  enable  integrated  modeling  of  aerodynamics 
and  structures  during  the  design  process.  The  tools  to 


do  this  are  being  developed  under  the  OSD  CRE- 
ATE-AV  program.  The  ability  to  integrate  these 
multiple  disciplines  will  address  many  of  the  subsystem 
issues  early  on  (i.e.,  the  subsystems  will  be  designed  for 
an  integrated  environment).  Having  advanced  diag¬ 
nostic  tools  such  as  PSP  in  ground-test  facilities  will 
not  only  enable  model  validation  but  will  also  better 
help  characterize  the  dynamic  flow  field  effects  on 
flight  vehicle  structures.  PSP  will  also  permit  rapid 
updating  of  flow  field  loads  as  part  of  structural 
analyses  without  having  to  build  or  update  pressure 
models.  As  discussed  earlier,  better  connectivity 
between  advanced  modeling  and  wind  tunnel  test¬ 
ing  needs  to  occur  early  and  often  prior  to  first  flight 
to  avoid  discovery  of  structural  issues  late  in  the 
program. 

Minimizing  potential  weight  growth  of  the  airframe 
structure  to  account  for  defects  discovered  in  flight  can 
also  have  an  important  effect  on  the  development  of 
the  propulsion  system.  Frequently,  when  weight 
growth  occurs  late  in  the  development  cycle  because 
of  structural  changes,  the  propulsion  system  developers 
are  tasked  to  produce  more  thrust  to  ensure  meeting 
vehicle  performance  parameters.  It  is  not  uncommon 
for  the  engine  developer  to  have  to  significantly 
improve  the  performance  of  the  engine  fairly  late  in 
the  development  cycle.  All  of  these  interactive  weight 
issues  also  impact  control  surface  effectiveness  and 
control  system  gains.  This  vicious  interplay  between 
the  various  subsystems  is  a  contributor  to  late  cycle 
churn  and  program  delays. 

There  is  also  the  potential  for  sharing  some  of  the 
same  modeling  methodologies  between  the  structural 
analysts  and  the  propulsion  system  designers.  The 
fluid-structure  interactions  that  drive  structural  design 
exhibit  the  same  fundamental  physics  as  the  fluid- 
structure  interactions  on  the  aeromechanics  of  fan  and 
compressor  blades.  Advances  in  integrated  CFD/CSD 
tools  will  help  better  understand  and  avoid  potential 
high-cycle  fatigue  issues  earlier  in  the  design  cycle. 

Detailed  modeling  of  a  turbine  engine  is  the  more 
challenging  of  the  CSE  capabilities  under  develop¬ 
ment.  With  hardware  and  software  advances  to  peta- 
flop  scales,  it  is  becoming  feasible  to  model  detailed 
rotating  machinery  such  as  the  first  stages  of  a  fan  on  a 
turbine  engine.  Coupled  with  a  detailed  model  of  the 
flow  field  around  the  aircraft  and  inlet,  it  will  be 
possible  to  model  the  integration  of  the  turbine  engine 
with  the  airframe  and  inlet  configuration.  Current 
capabilities  and  practices  design  and  develop  the  engine 
independent  of  the  airframe  and  try  to  account  for 
integration  issues  discovered  during  testing.  Frequently 
these  issues  are  not  fully  discovered  until  after  first 
flight. 
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The  discussions  in  this  section  present  an  aggressive 
use  and  integration  of  modeling  and  testing  simulation 
methodologies  to  change  the  future  effectiveness  of 
aeronautical  development.  It  is  clear  that  various  test 
capabilities  cannot  be  addressed  and  judged  in  isolation 
but  have  to  be  treated  as  an  integral  combination  with 
technical  expertise,  improved  processes,  and  better  test 
methods  to  achieve  the  desired  state  of  effectiveness.  In 
addition,  they  will  have  to  be  applied  in  a  common 
environment  to  ensure  gains  in  effectiveness  can  be 
replicated  from  program  to  program. 

Conclusions 

High-performance  computing  has  advanced  to  a 
state  that  should  support  more  applications  of  CSE  in 
the  aeronautical  system  development  process.  With 
such  advances,  a  national  debate  has  reemerged  on 
using  CSE  to  replace  testing.  The  author  argues  that  a 
national  discussion  of  replacing  testing  with  CSE  is 
misguided.  The  nation  would  be  better  served  by 
putting  our  energy  into  determining  approaches  to 
fully  integrate  CSE  with  testing  to  reduce  the  cycle 
time  for  aeronautical  system  development.  To  success¬ 
fully  integrate  CSE  and  testing  will  require  advances 
not  only  in  high-performance  computing  but  in 
intellectual  capital  and  process  management  as  well. 
Key  recommendations  for  advancing  the  use  of  CSE 
are  as  follows: 

•  Most  important,  the  government  has  to  adopt  a 
monopsony  for  the  application  of  CSE  to  the 
development  process  for  military  flight  systems. 

•  A  common  architecture  for  the  application  of 
multidisciplinary  computational  tools  in  a  high- 
performance  computing  environment  needs  to  be 
adopted  by  the  industry.  This  architecture  should 
not  preclude  use  of  proprietary  physical  models 
from  industry  but  should  enable  CSE  and  testing 
to  be  optimized  for  use  across  any  aeronautical 
development  process. 

•  In  spite  of  the  computer  hardware  systems 
advances,  there  is  still  much  work  to  be  done  in 
building  the  software  tools  to  best  use  the 
advanced  computer  systems;  notably,  better 
physics  modeling,  scalability  of  solvers  to  tens  of 
thousands  of  processors,  and  better  multidisci¬ 
plinary  modeling  to  enable  dynamic  simulation  of 
complete  maneuvering  aircraft. 

•  CSE  alone  will  not  provide  maximum  impact  to 
cycle  time  reduction  but  must  be  integrated  with 
other  tools  such  as  design  of  experiments, 
streamlined  test  methodologies,  advanced  diag¬ 
nostic  tools,  networking,  and  knowledge  man¬ 
agement. 


•  In  addition,  a  concept  of  operations  and  the 
necessary  computing  capacity  need  to  be  devel¬ 
oped  to  support  the  aeronautical  systems  engi¬ 
neering  process.  □ 
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